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1. BACKGROUND 

The TRMM data system has been providing near-realtime data products to the community since late 1999. While 
the TRMM project never had near-realtime production requirements, the science and applications communities 
had a great interest in receiving TRMM data as quickly as possible. As a result these NRT data are provided under 
a best-effort scenario but with the objective of having the swath data products available within three hours of data 
collection 90% of the time. In July of 2011 the Joint Precipitation Measurement Missions Science Team (JPST) 
authorized the reprocessing of TRMM mission data using the new version 7 algorithms. The reprocessing of the 
14+ years of the mission was concluded within 30 days. 

Version 7 algorithms had substantial changes in the data product file formats both for data and metadata. In 
addition, the algorithms themselves had major modifications and improvements. The general approach to 
versioning up the NRT is to wait for the regular production algorithms to have run for a while and shake out any 
issues that might arise from the new version before updating the NRT products. Because of the substantial 
changes in data/metadata formats as well as the algorithm improvements themselves, the update of NRT to V7 
followed an even more conservative path than usual. This was done to ensure that applications agencies and 
other users of the TRMM NRT would not be faces with short -timeframes for conversion to the new format. 

This paper will describe the process by which the TRMM NRT was updated to V7 and the V7 data products 
themselves. 


2. TRMM V7 NRT UPDATE APPROACH AND PROCESSING ENVIRONMENT 
2.1 Update Approach 

The TRMM NRT algorithms are always updated after the production version of the algorithms have run for a 
while and shown themselves to be stable and without problems. The basic approach is to allow the new algorithm 
version to have run for at least 6 months before the NRT algorithms are updated. The same approach was the 



basis for the V7 update. Generally, once the new NRT version has shown itself stable in the NRT system the 
previous version is stopped and the new version becomes operational. 

NRT V7 has many changes in data format and metadata format [3]. While the data formats had large changes, the 
metadata was a substantial redesign in V7 over V6. It was clear that additional time would need to be provided to 
users to make and the necessary changes in their software. As a result it was decided that for a period of up to six 
months, the V6 and the V7 NRT products would be available in parallel to users. As a result for the first time the 
name of the TRMM NRT products were altered and a version number was included in the version 7 products. 

An important consideration during the design phase of the TRMM NRT was to create products that contain all the 
required parameters for proper use but still small enough for easy and fast network access. So, most of the 
diagnostic and supplementary parameters were removed from NRT products. In addition, NRT products generally 
used scaled fields to further reduce their sizes. In previous NRT versions the TM1 GPROF retrievals were run 
with a special algorithm in the NRT. This required the algorithm developer to check science content for two 
separate code packages. For V7 NRT the TMI retrieval approach is the same as for the PR. The following two 
step process is used: 

a. The regular V7 algorithms are run and the full V7 products are produced. In the case of the GPROF 
retrievals the vertical structure generation is turned off in NRT. 

b. The regular products produced are then parameter subsetted to create the actual realtime products. 

This two step approach guarantees that a single “validated” production algorithm is used, that parameter 
subsetting does not alter the values but still allows smaller file size, and allows isolation between formats of the 
production products and the NRT products. In NRT V7 all the algorithms follow this two step process. This has 
allowed the V7 NRT products to stay closer to their V7 counterparts than was possible with the regular TRMM 
products. 

2.2 Processing Environment 

Since the beginning of the TRMM NRT production, all processing was done on an SGI under the IRIX operating 
system. However, with the V7 NRT system, the hardware environment totally changed and this change also had a 
major impact on the time required for converting from V6 to V7. 

The TRMM NRT V7 is run on a Penguin Beowulf cluster. This machine has redundant host nodes and 20 
processing nodes. The data is stored on a Panasas clustered file server. The Scyld Beowulf operating system is 



used. This is based on CentOS version 5 (an open source Redhat Enterprise compatible operating system). This 
hardware environment is substantially faster than the old SGI environment. As a result even though the algorithms 
require more processing capacity and an extra step was added for the TMI retrievals, the new V7 NRT completes 
its processing more quickly than V6 NRT. 

This change from a multi-processor board environment under control of IRIX to a multiple processor node 
environment required substantial modification in the code that schedules the flow of the NRT processing. Also 
because of multiple processors accessing the same file systems at the same time, an effective clustered file system 
had to be provided. Panasas has successfully provided safe clustered access for the regular TRMM production for 
quite a while and is therefore key to the entire processing strategy on the V7 NRT. 


The changes from an SGI/IRIX environment to a Penguin Beowulf/Scyld environment does have some impact on 
values generally through round-off issues, etc. However, it has been demonstrated that these round-off issues do 
not affect the accuracy of the calculations. Indeed, the new environment provides more accuracy and precision 
than was possible in the SGI new-32bit processing environment. 


3. DATA PRODUCTS 

3.1 Background 

An important difference between the TRMM V7 NRT products and the TRMM regular products is that predictive 
empheris is used for geolocation rather than the definitive ephemeris. In addition geolocation in NRT products is 
scaled but this still keeps it within the 2.5km geolocation accuracy. Lastly many parameters that are stored as 
floats in the TRMM V7 products are stored as scaled 2 byte integers in the TRMM NRT V7. 


V7 NRT products also have a minimum of metadata data again as part of the effort to keep the product size small. 

3.2 Visible Infrared Scanner (VIRS) 

Only a single VIRS product is produced and this is the LIB radiance product. This is the full V7 product but with 
the restrictions listed in paragraph 3.1 above. [1] 

3.3 TRMM Microwave Imager (TMI) 

Two primary NRT V7 products are produced for TMI ['1][2J. The Tb product at LIB and the GPROL 2010 
precipitation retrieval product at Level 2. The GPROL retrieval is parameter subsetted from the full product to the 
NRT product [4] The NRT GPROL retrievals do not include vertical structure information and have some scaled 
2 byte integer values where the production products have float values. 



A full version of the LIB and the GPROF are available via the NRT system for those who want the larger files 
and don’t have network access issues. 


3.4 Precipitation Radar (PR) 

Three major PR products are available through the V7 NRT system. A rain classification product that also 
includes brightband information (2A23rt) and 2 different precipitation retrievals. The first 2A25rl includes the 
estimated surface precipitation and the near-surface reflectivity and the 2 nd is an 25 vertical level (rather than full 
80 level) rain retrieval and reflectivity. As always these products are parameter subsetted versions [4] of the full 
products [1], 
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TRMM V7 NRT Background 




* TRMM NRT V7 swath products: VIRS, TMI, PR were 
available for testing beginning in October 2011 

* TRMM NRT V7 swath products became the 
operational version on 1 December 2011 

* TRMM NRT V6 swath products were produced in 
parallel until 30 June 2012 

* TRMM Multiple-satellite Precipitation Analysis TMPA 
NRT V7 available for testing in April 2012 

* TRMM NRT V7 TMPA became operational 25 June 
2012 

* Currently only V7 NRT is operational 
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TRMM NRT Hardware Environment 




* TRMM NRT V6 was run on an SGI Origin 3000 under 
the IRIX operating system. The only SGI left at PPS 

* TRMM NRT V7 is run on a Beowulf clustered parallel 
processing environment 

- 20 AMD Quad-core processing nodes (each with 16 GB 
of memory) 

-2 Host nodes (1 primary, 1 warm backup) 

- Scyld operating system (based on Linux CentOS 5) 

- 12 TB of Panasas high-speed clustered disk storage 
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TRMM NRT V7 Swath Products 




• NRT Swath products are not technically orbital they are based on a 
TDRSS scheduled access (SA) contact 

- Mostly about ~93 minutes 

- However, because of scheduling conflicts could be >~1 10 minutes 

- Also could be less than ~93 minutes 

• VIRS - 1B01 RT radiance product at IFOV 

• TMI - 1 B1 1 RT brightness temperature (Tb) product at IFOV 

• GPROF - 2A12 RT TMI precipitation retrieval at IFOV 

• PR- 2A25r1 RT near surface PR precipitation retrieval at IFOV 

• PR-2A25r2 RT 20 vertical level PR precipitation retrieval at IFOV 
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Major Difference V6 and V7 




• 1 B 1 1 rt includes the solar bias correction introduced in Ml 


Substantial metadata changes between V6 and Ml 


• 1 B1 1 rt M6->M1 changes 

V6 

• #scans in file meta 

• ScanTime Vdata table 

• Geolocation 

• 

• 2A12rt V6->V7 changes 

V6 

• #scans in file meta 

• ScanTime Vdata table 
. Geolocation 


Rain Flag 


V7 

• #scans in swath meta 

• SDS array 

• Latitude, Longitude 
. scanStatus 


V7 

. #scans in swath meta 

. ScanTime SDS array 

. Latitude, Longitude 
. ScanStatus 
. SurfaceType 
. ProbabilityOfPrecipitation 
. QualityFlag, pixelStatus, landScreenFlag, 
landAmbiguousFlag 
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2A12 (GPROF) V6 -V7 Differences (2) PPS 



V6 

. SurfaceRain 
. ConvectiveSurfaceRain 

• Total Cloud Liquid Water 

• Total Precipitation Ice 


V7 

. SurfacePrecipitation 
. ConvectPrecipitation 


In V7 CLW and TPI are calculated as part of calculations for vertical 
structure. Because of the extra time for this processing (sometimes very 
long) and the latency requirements of NRT, the vertical structure code is not 
activated 
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TRMM NRTV6 1B11 
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TRMM NRTV7 1B11 
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TRMM V6 GPROF NRT (2A12) PPS 
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TRMM V7 GPROF2010 NRT (2A12) 
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TRMMTMPA V7 NRT 




• Three products are available: 

- 3B40rt - merged radiometer product 3-hourly product 

- 3B41rt - calibrated (by radiometers) IR 1-hour product 

- 3B42rt - 3-hourly radiometer product filled in by IR when no radiometer 
available 

• These products are in binary format. No format changes between V6 
and Ml 

• Ml radiometer products include TMI, SSMIS (FI 6, F17, F18) and MHS 
from NOAA and Metop series 

• Have a TIFF (scaled surface rain) with World File (boundaries) 
3B42RT output usable with GIS software 

- 3 HR 

- 1 day accumulation 

- 7 day accumulation 
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Access To TRMM NRT V7 



• Swath Products 

- trmmrt.gsfc.nasa.gov 

- User: trmmreal 

- Password: your email 

- Anonymous ftp 

- Subdirectories: VIRS, PR, TMI, documentation 

• TMPA realtime 

- trmmopen.gsfc.nasa.gov 

- User: anonymous 

- Password: your email 

- Anonymous ftp 

- TMPA: pub/merged/calibratedIR, pub/merged/mergeIRMicro, 
pub/merged/combinedMicro 

- TMPA GIS: pub/gis 
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TMPA V7 NRT Retrieval Online 



’TRMM Multi-satellite Precipitation A 


File Edit View History Bookmarks loots 


# pps .gsfc . nasa .gov/tmpa/?di rectory; 




National Aeronautics 
and Space Administration 


+ PPS Home 
+ Helpdesk 



TRMM Multi-satellite Precipitation Analysis (TMPA) 

Home / TRMMrealtime / mergeIRMicro / 2012 1 07 1 04 

GIS TIF files are NOT viewable as images. Download if desired. 

To download a text or TIF file “Ctrl + Click" (Mac) or "Right Click" (Other platforms) 


Product 

Type 

HOUR 

Download / 
View 

TMPA files 

GIS 

3-HOUR 

GIS 

1-Day 

GIS 

3-Day 

GIS 
7 -Day 

3B42RT 

00 

li Tl+AR 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

03 

If THAr 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

06 

Mi Thar 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

09 

ML Thar 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

12 

!i THAr 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

15 

Wi THAr 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

18 

M. Thar 

tif tfw 

tif tfw 

tif tfw 

tif tfw 

3B42RT 

21 

ill THAr 

tif tfw 

tif tfw 

tif tfw 

tif tfw 
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TMPA NRT V 7 View Online 





pps.gsfc.nasa.gov/tmpa/ 
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